<!DOCTYPE html>
<html xmlns:wb="http://open.weibo.com/wb">
<head>
  <meta charset="utf-8">
  <script src="https://cdn.jsdelivr.net/gh/Sanarous/files@1.13/js/linkcard.js"></script>
  <script>
(function(){
    var bp = document.createElement('script');
    var curProtocol = window.location.protocol.split(':')[0];
    if (curProtocol === 'https') {
        bp.src = 'https://zz.bdstatic.com/linksubmit/push.js';
    }
    else {
        bp.src = 'http://push.zhanzhang.baidu.com/push.js';
    }
    var s = document.getElementsByTagName("script")[0];
    s.parentNode.insertBefore(bp, s);
})();
</script>
<script>
var _hmt = _hmt || [];
(function() {
  var hm = document.createElement("script");
  hm.src = "https://hm.baidu.com/hm.js?fc9a8559a133f4d8ce784d69d6337bb0";
  var s = document.getElementsByTagName("script")[0]; 
  s.parentNode.insertBefore(hm, s);
})();
</script>

  
  <title>flink on yarn集群搭建及验证要点记录 | 涂宗勋的博客</title>
  <meta name="baidu-site-verification" content="o8pWlgAEZ7" />
  <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">
  <meta name="description" content="standalone模式的弊端前面搭建了flink简单集群，并在此基础上又搭建了HA集群，记录地址如下： flink初识及集群搭建和简单验证 flink-HA集群搭建和问题记录 虽然上述两种都能使用，在学习flink-api阶段应该是够用了，但是如果真要上生产使用，就还是有一定的弊端。">
<meta property="og:type" content="article">
<meta property="og:title" content="flink on yarn集群搭建及验证要点记录">
<meta property="og:url" content="https://tuzongxun.gitee.io/2020/10/16/flink3/index.html">
<meta property="og:site_name" content="涂宗勋的博客">
<meta property="og:description" content="standalone模式的弊端前面搭建了flink简单集群，并在此基础上又搭建了HA集群，记录地址如下： flink初识及集群搭建和简单验证 flink-HA集群搭建和问题记录 虽然上述两种都能使用，在学习flink-api阶段应该是够用了，但是如果真要上生产使用，就还是有一定的弊端。">
<meta property="og:locale" content="zh_CN">
<meta property="article:published_time" content="2020-10-16T04:59:42.000Z">
<meta property="article:modified_time" content="2020-10-16T03:41:38.369Z">
<meta property="article:author" content="涂宗勋">
<meta property="article:tag" content="flink">
<meta name="twitter:card" content="summary">
  
  
    <link rel="icon" href="/images/touxiang.png">
  
  
    
  
  
<link rel="stylesheet" href="/tzxblog/css/style.css">

  

<meta name="generator" content="Hexo 4.2.1"></head>

<body>
  <div id="container">
    <div id="wrap">
      <header id="header">
  <script src="https://tjs.sjs.sinajs.cn/open/api/js/wb.js" type="text/javascript" charset="utf-8"></script>
  <script src="https://cdn.jsdelivr.net/gh/Sanarous/files@1.13/js/linkcard.js"></script>
  <div id="banner"></div>
  <div id="header-outer" class="outer">
    
    <div id="header-inner" class="inner">
      <nav id="sub-nav">
        
        <a id="nav-search-btn" class="nav-icon" title="搜索"></a>
      </nav>
      <div id="search-form-wrap">
        <form action="//google.com/search" method="get" accept-charset="UTF-8" class="search-form"><input type="search" name="q" class="search-form-input" placeholder="Search"><button type="submit" class="search-form-submit">&#xF002;</button><input type="hidden" name="sitesearch" value="https://tuzongxun.gitee.io"></form>
      </div>
      <nav id="main-nav">
        <a id="main-nav-toggle" class="nav-icon"></a>
        
          <a class="main-nav-link" href="/tzxblog/">首页</a>
        
          <a class="main-nav-link" href="/tzxblog/shuoshuo/">说说</a>
        
          <a class="main-nav-link" href="/tzxblog/archives/">归档</a>
        
          <a class="main-nav-link" href="/tzxblog/collections/">导航</a>
        
          <a class="main-nav-link" href="/tzxblog/download/">资源</a>
        
          <a class="main-nav-link" href="/tzxblog/about/">简历</a>
        
      </nav>
      
    </div>
    <div id="header-title" class="inner">
      <h1 id="logo-wrap">
        <a href="/tzxblog/" id="logo">涂宗勋的博客</a>
      </h1>
      
        <h2 id="subtitle-wrap">
          <a href="/tzxblog/" id="subtitle">java程序员，现居武汉，CSDN博客https://blog.csdn.net/tuzongxun</a>&nbsp;&nbsp;&nbsp;&nbsp;
		  <!--<span id="busuanzi_container_site_pv">【本站累计访问量:<span id="busuanzi_value_site_pv"></span>】</span>-->
        </h2>
		
      
    </div>
  </div>
</header>
      <div class="outer">
        <section id="main"><article id="post-flink3" class="article article-type-post" itemscope itemprop="blogPost">
  <div class="article-meta">
    <a href="/tzxblog/2020/10/16/flink3/" class="article-date">
  <time datetime="2020-10-16T04:59:42.000Z" itemprop="datePublished">2020-10-16</time>
</a>
    
  <div class="article-category">
    <a class="article-category-link" href="/tzxblog/categories/flink/">flink</a>
  </div>

</span>
  </div>
  <div class="article-inner">
    
    
      <header class="article-header">
        
  
    <h1 class="article-title" itemprop="name">
      flink on yarn集群搭建及验证要点记录
    </h1>
  

      </header>
    
    <div class="article-entry" itemprop="articleBody">
      
        <!-- Table of Contents -->
        
        <h2 id="standalone模式的弊端"><a href="#standalone模式的弊端" class="headerlink" title="standalone模式的弊端"></a>standalone模式的弊端</h2><p>前面搭建了flink简单集群，并在此基础上又搭建了HA集群，记录地址如下：</p>
<p><a href="https://tuzongxun.blog.csdn.net/article/details/109039580" target="_blank" rel="noopener">flink初识及集群搭建和简单验证</a></p>
<p><a href="https://tuzongxun.blog.csdn.net/article/details/109081839" target="_blank" rel="noopener">flink-HA集群搭建和问题记录</a></p>
<p>虽然上述两种都能使用，在学习flink-api阶段应该是够用了，但是如果真要上生产使用，就还是有一定的弊端。</p>
<a id="more"></a>

<p>根据之前的学习可知，flink集群主要分为<code>jobManager</code>和<code>taskManager</code>，而jobManger的任务主要有两个，一个是<code>资源管理</code>，另一个是<code>任务调度</code>。</p>
<p>这样一来，jobManager的任务其实就显得有点多，而又由于mapReduce、spark、flink的集群都能基于yarn管理资源，所以有一个更好的方式就是让flink集群运行在yarn上。</p>
<p>yarn是hadoop项目自带的模块，启动hadoop集群会一起启动yarn，所以也不用单纯的再维护yarn，同时，yarn的资源管理也能分担jobManager的资源管理，使得这种模式下的jobManager基本只需要专注于任务调度，也能进一步提高可用性。</p>
<p>之所以说是基本专注，是因为它不直接管理资源，但是还是要为taskManager向yarn申请资源，而这里说的yarn管理资源，实际上就是之前hdfs搭建时里边所说的<code>resourceManager</code>，可以参考之前的hdfs搭建记录：</p>
<p><a href="https://tuzongxun.blog.csdn.net/article/details/108347899" target="_blank" rel="noopener">HDFS-HA模式搭建（基于完全分布式模式升级）</a>及<a href="https://tuzongxun.blog.csdn.net/article/details/107843326" target="_blank" rel="noopener">hadoop分布式安装及配置初步解析（坑坑不息）</a></p>
<h2 id="flink-on-yarn运行"><a href="#flink-on-yarn运行" class="headerlink" title="flink on yarn运行"></a>flink on yarn运行</h2><p>单纯的使flink集群运行在yarn上，不需要额外的配置，只需要使用yarn命令启动flink集群即可，但是使用方式有两种，一种是提交任务的同时启动集群，另一种是先启动集群再提交运行任务。</p>
<p>前一种方式每次提供运行任务时启动集群，关闭任务也会关了集群，后一种则是一直保持<code>jobManager</code>开启，<code>taskManager</code>按需启动，因此后一种相对更加常用，这种方式分为两步：</p>
<p>步骤一，在yarn上启动flink集群，例如：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">yarn-session.sh -n 3 -s 3 -nm flink-yarn-test -d</span><br></pre></td></tr></table></figure>

<p>上边参数的意思是：<code>-n</code>代表最多启动的<code>taskManager</code>数量，<code>-s</code>代表每个<code>taskManager</code>中最多分配的<code>slot</code>数量，<code>-nm</code>代表自定义的flink集群名称，<code>-d</code>代表后台运行。</p>
<p>步骤二，提交flink任务，这里其实又可以分为两种方式，一种是在web界面提交，一种是命令行，命令行提交示例如下：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">flink run -c com.tzx.study.demo.flink.FlinkTest -yid application_1586794520478_0007 tzx-study-demo.jar</span><br></pre></td></tr></table></figure>

<p>这里<code>-c</code>指定flink程序启动类的路径，<code>-yid</code>指定任务要提交到的flink（yarn）集群的id，末尾是jar包路径，我是在jar包所在目录执行的命令，因此就没有其他前缀。</p>
<p>这里需要注意的是，<code>-yid</code>可以不指定，默认会提交到最新启动的flink(yarn)集群中。</p>
<p><strong>注：上述操作若有问题，请继续往下看。上述操作是理论操作，因为HA方式也是一样，因此我直接在HA中操作，因此有些问题会直接记录在下边。</strong></p>
<h2 id="flink-on-yarn-HA配置"><a href="#flink-on-yarn-HA配置" class="headerlink" title="flink on yarn HA配置"></a>flink on yarn HA配置</h2><p>上边的方式把flink运行在yarn上，有一个问题在于<code>jobManager</code>一样是单机的，也会有单点故障，因此正常的生产环境也应该是要使用HA方式。</p>
<p>yarn中的flink的HA，实际是增加<code>jobManager</code>的故障重试次数，进而使得原本运行的<code>jobManager</code>出现问题后，yarn能够再启动一个新的<code>jobManager</code>，从而提高整个flink集群的可用性，这个配置在<code>hadoop</code>中的<code>yarn-site.xml</code>文件中，增加如下配置：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">&lt;property&gt;</span><br><span class="line">        &lt;name&gt;yarn.resourcemanager.am.max-attempts&lt;&#x2F;name&gt;</span><br><span class="line">        &lt;value&gt;10&lt;&#x2F;value&gt;</span><br><span class="line">&lt;&#x2F;property&gt;</span><br></pre></td></tr></table></figure>

<p>有了上述配置之后，一次启动zookeeper集群、hdfs集群、flink集群，但是执行<code>yarn-session</code>启动flink集群时却出错了，错误如下：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br></pre></td><td class="code"><pre><span class="line">2020-10-16 09:52:05,326 ERROR org.apache.flink.yarn.cli.FlinkYarnSessionCli                [] - Error while running the Flink session.</span><br><span class="line">org.apache.flink.client.deployment.ClusterDeploymentException: Couldn&#39;t deploy Yarn session cluster</span><br><span class="line">	at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:382) ~[flink-dist_2.11-1.11.2.jar:1.11.2]</span><br><span class="line">	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:514) ~[flink-dist_2.11-1.11.2.jar:1.11.2]</span><br><span class="line">	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.lambda$main$4(FlinkYarnSessionCli.java:751) ~[flink-dist_2.11-1.11.2.jar:1.11.2]</span><br><span class="line">	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_261]</span><br><span class="line">	at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_261]</span><br><span class="line">	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836) ~[flink-shaded-hadoop-2-uber-2.8.3-10.0.jar:2.8.3-10.0]</span><br><span class="line">	at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41) ~[flink-dist_2.11-1.11.2.jar:1.11.2]</span><br><span class="line">	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:751) [flink-dist_2.11-1.11.2.jar:1.11.2]</span><br><span class="line">Caused by: org.apache.flink.yarn.YarnClusterDescriptor$YarnDeploymentException: The YARN application unexpectedly switched to state FAILED during deployment. </span><br><span class="line">Diagnostics from YARN: Application application_1602812938780_0001 failed 2 times in previous 10000 milliseconds (global limit &#x3D;10; local limit is &#x3D;2) due to AM Container for appattempt_1602812938780_0001_000003 exited with  exitCode: 127</span><br><span class="line">Failing this attempt.Diagnostics: [2020-10-16 09:52:04.913]Exception from container-launch.</span><br><span class="line">Container id: container_1602812938780_0001_03_000001</span><br><span class="line">Exit code: 127</span><br><span class="line"></span><br><span class="line">[2020-10-16 09:52:04.920]Container exited with a non-zero exit code 127. Error file: prelaunch.err.</span><br><span class="line">Last 4096 bytes of prelaunch.err :</span><br><span class="line"></span><br><span class="line">[2020-10-16 09:52:04.931]Container exited with a non-zero exit code 127. Error file: prelaunch.err.</span><br><span class="line">Last 4096 bytes of prelaunch.err :</span><br><span class="line"></span><br><span class="line">For more detailed output, check the application tracking page: http:&#x2F;&#x2F;node001:18088&#x2F;cluster&#x2F;app&#x2F;application_1602812938780_0001 Then click on links to logs of each attempt.</span><br><span class="line">. Failing the application.</span><br><span class="line">If log aggregation is enabled on your cluster, use this command to further investigate the issue:</span><br><span class="line">yarn logs -applicationId application_1602812938780_0001</span><br><span class="line">	at org.apache.flink.yarn.YarnClusterDescriptor.startAppMaster(YarnClusterDescriptor.java:1021) ~[flink-dist_2.11-1.11.2.jar:1.11.2]</span><br><span class="line">	at org.apache.flink.yarn.YarnClusterDescriptor.deployInternal(YarnClusterDescriptor.java:524) ~[flink-dist_2.11-1.11.2.jar:1.11.2]</span><br><span class="line">	at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:375) ~[flink-dist_2.11-1.11.2.jar:1.11.2]</span><br><span class="line">	... 7 more</span><br></pre></td></tr></table></figure>

<p>上述提示内容没有看出实质性的问题，但是却发现了如下两行：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">If log aggregation is enabled on your cluster, use this command to further investigate the issue:</span><br><span class="line">yarn logs -applicationId application_1602812938780_0001</span><br></pre></td></tr></table></figure>

<p>不用完全看懂也能大概知道要进一步查问题，需要执行<code>yarn logs -applicationId application_1602812938780_0001</code>命令，但是直接执行会发现也看不到什么内容，反而有如下错误提示：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">2020-10-16 10:07:48,478 INFO client.RMProxy: Connecting to ResourceManager at node001&#x2F;192.168.139.91:18040</span><br><span class="line">File &#x2F;tmp&#x2F;logs&#x2F;root&#x2F;logs-tfile&#x2F;application_1602812938780_0001 does not exist.</span><br><span class="line"></span><br><span class="line">Can not find any log file matching the pattern: [ALL] for the application: application_1602813970991_0001</span><br><span class="line">Can not find the logs for the application: application_1602813970991_0001 with the appOwner: root</span><br></pre></td></tr></table></figure>

<p>这个问题其实上边有说明了，使用上述命令的前提是集群中开启聚合日志，因此需要在<code>yarn-site.xml</code>中再增加一个配置开启这个日志：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">&lt;property&gt;</span><br><span class="line">    &lt;name&gt;yarn.log-aggregation-enable&lt;&#x2F;name&gt;</span><br><span class="line">    &lt;value&gt;true&lt;&#x2F;value&gt;</span><br><span class="line">&lt;&#x2F;property&gt;</span><br></pre></td></tr></table></figure>

<p>重启hdfs集群后，再执行<code>yarn-session</code>来启动flink集群，还是会出现启动失败的提示，但是再次执行<code>yarn logs</code>就有了更多的日志信息，其中有一段如下：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">Container: container_1602813970991_0001_02_000001 on node002_40809</span><br><span class="line">LogAggregationType: AGGREGATED</span><br><span class="line">&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;&#x3D;</span><br><span class="line">LogType:jobmanager.err</span><br><span class="line">LogLastModifiedTime:Fri Oct 16 09:52:05 +0800 2020</span><br><span class="line">LogLength:48</span><br><span class="line">LogContents:</span><br><span class="line">&#x2F;bin&#x2F;bash: &#x2F;bin&#x2F;java: No such file or directory</span><br><span class="line"></span><br><span class="line">End of LogType:jobmanager.err</span><br></pre></td></tr></table></figure>

<p>有了这段提示，原因就显而易见了，这是很多地方的常见问题，找不到<code>JAVA_HOME</code>。</p>
<p>但是为啥找不到呢，联想一下之前的hdfs集群搭建，想起来hdfs集群启动的时候，会从一台机<code>ssh</code>到另一台机启动shell进而启动该节点。</p>
<p>而这个shell不会加载<code>/etc/profile</code>文件，也就导致会找不到<code>JAVA_HOME</code>。</p>
<p>根据之前在<code>hadoop-env.sh</code>中增加<code>JAVA_HOME</code>配置的经验，猜想应该是yarn需要一样的处理，于是在<code>yarn-env.sh</code>中加入了<code>JAVA_HOME</code>的配置：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">export JAVA_HOME&#x3D;&#x2F;opt&#x2F;java&#x2F;jdk1.8.0_261</span><br></pre></td></tr></table></figure>

<p><strong>需要注意，以上所有配置的修改，均需要同步分发到所有节点。</strong></p>
<h2 id="web-ui"><a href="#web-ui" class="headerlink" title="web ui"></a>web ui</h2><p>之前在<code>standalone</code>模式中，可以直接访问<code>8081</code>端口查看<code>jobManager</code>的<code>web ui</code>，在这里界面能看到很多信息，也能直接提交flink的任务，而flink运行在yarn上，一样有<code>web ui</code>，只不过有两个。</p>
<p>其中一个可以通过<code>yarn</code>的<code>web ui</code>跳转，另一个则是直接访问，但是这两个有所区别。</p>
<p>通过<code>yarn</code>跳转的，默认只能看一些信息，不能提交任务，而直接访问的，和原来<code>8081</code>访问的效果一样。</p>
<p>上边配置好<code>JAVA_HOME</code>后，再重启hadoop和yarn下的flink集群，会看到不再报错，并且最后带出了<code>jobManager</code>的ui地址：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">2020-10-16 10:33:15,488 INFO  org.apache.flink.shaded.curator4.org.apache.curator.framework.state.ConnectionStateManager [] - State change: CONNECTED</span><br><span class="line">JobManager Web Interface: http:&#x2F;&#x2F;node003:37917</span><br></pre></td></tr></table></figure>

<p>上边这个<code>http://node003:37917</code>就是我现在可直接访问的，等同之前<code>8081</code>访问的<code>jobManager</code>的<code>web ui</code>。</p>
<p>而使用<code>yarn</code>跳转方式，需要先访问yarn的<code>web ui</code>，默认端口是<code>8088</code>，而我搭建的时候配置的是<code>18088</code>。</p>
<h2 id="其他要点"><a href="#其他要点" class="headerlink" title="其他要点"></a>其他要点</h2><p>在<code>standalone</code>模式的HA集群中，有两个重要文件，<code>masters</code>和<code>workers</code>，这两个文件决定了<code>jobManager</code>和<code>taskManager</code>节点。</p>
<p>而在yarn中，资源是yarn管理，这两个文件实际是无效的，我试过清空这两个文件的内容，对运行没有任何影响。</p>
<p>但是和<code>standalone</code>模式一样的是，在系统环境变量中依然需要有下边的配置存在：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">export HADOOP_CONF_DIR&#x3D;$HADOOP_HOME&#x2F;etc&#x2F;hadoop export </span><br><span class="line"></span><br><span class="line">HADOOP_CLASSPATH&#x3D;&#96;hadoop classpath&#96;</span><br></pre></td></tr></table></figure>


      
    </div>
    <footer class="article-footer">
      <a data-url="https://tuzongxun.gitee.io/2020/10/16/flink3/" data-id="ckxn7cxg5001ckcvh746recl5" class="article-share-link">分享</a>
      
      
      
  <ul class="article-tag-list" itemprop="keywords"><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tzxblog/tags/flink/" rel="tag">flink</a></li></ul>

    </footer>
  </div>
  
    
  <div class="comments" id="comments">
    
     
       
       
      
      
	 
  </div>
 
    
 
<script src="/tzxblog/jquery/jquery.min.js"></script>

  <div id="random_posts">
    <h2>推荐文章</h2>
    <div class="random_posts_ul">
      <script>
          var random_count =5
          var site = {BASE_URI:'/tzxblog/'};
          function load_random_posts(obj) {
              var arr=site.posts;
              if (!obj) return;
              // var count = $(obj).attr('data-count') || 6;
              for (var i, tmp, n = arr.length; n; i = Math.floor(Math.random() * n), tmp = arr[--n], arr[n] = arr[i], arr[i] = tmp);
              arr = arr.slice(0, random_count);
              var html = '<ul>';
            
              for(var j=0;j<arr.length;j++){
                var item=arr[j];
                html += '<li><strong>' + 
                item.date + ':&nbsp;&nbsp;<a href="' + (site.BASE_URI+item.uri) + '">' + 
                (item.title || item.uri) + '</a></strong>';
                if(item.excerpt){
                  html +='<div class="post-excerpt">'+item.excerpt+'</div>';
                }
                html +='</li>';
                
              }
              $(obj).html(html + '</ul>');
          }
          $('.random_posts_ul').each(function () {
              var c = this;
              if (!site.posts || !site.posts.length){
                  $.getJSON(site.BASE_URI + 'js/posts.js',function(json){site.posts = json;load_random_posts(c)});
              } 
               else{
                load_random_posts(c);
              }
          });
      </script>
    </div>
  </div>

	
<nav id="article-nav">
  
    <a href="/tzxblog/2020/10/23/wenti3/" id="article-nav-newer" class="article-nav-link-wrap">
      <strong class="article-nav-caption">上一篇</strong>
      <div class="article-nav-title">
        
          近期生产问题和解决方案记录
        
      </div>
    </a>
  
  
    <a href="/tzxblog/2020/10/14/flink2/" id="article-nav-older" class="article-nav-link-wrap">
      <strong class="article-nav-caption">下一篇</strong>
      <div class="article-nav-title">flink-HA集群搭建和问题记录</div>
    </a>
  
</nav>

  
</article>

</section>
           
    <aside id="sidebar">
  
    <!--微信公众号二维码-->


  
    

  
    
  
    
    <div class="widget-wrap">
    
      <div class="widget" id="toc-widget-fixed">
      
        <strong class="toc-title">文章目录</strong>
        <div class="toc-widget-list">
              <ol class="toc"><li class="toc-item toc-level-2"><a class="toc-link" href="#standalone模式的弊端"><span class="toc-number">1.</span> <span class="toc-text">standalone模式的弊端</span></a></li><li class="toc-item toc-level-2"><a class="toc-link" href="#flink-on-yarn运行"><span class="toc-number">2.</span> <span class="toc-text">flink on yarn运行</span></a></li><li class="toc-item toc-level-2"><a class="toc-link" href="#flink-on-yarn-HA配置"><span class="toc-number">3.</span> <span class="toc-text">flink on yarn HA配置</span></a></li><li class="toc-item toc-level-2"><a class="toc-link" href="#web-ui"><span class="toc-number">4.</span> <span class="toc-text">web ui</span></a></li><li class="toc-item toc-level-2"><a class="toc-link" href="#其他要点"><span class="toc-number">5.</span> <span class="toc-text">其他要点</span></a></li></ol>
          </div>
      </div>
    </div>

  
    

  
    
  
    
  
    

  
</aside>

      </div>
      <footer id="footer">
  <script async src="//busuanzi.ibruce.info/busuanzi/2.3/busuanzi.pure.mini.js"></script>
  
  <div class="outer">
    <div id="footer-left">
      &copy; 2016 - 2021 涂宗勋&nbsp; <a href="https://beian.miit.gov.cn/#/Integrated/recordQuery" target="_blank" rel="noopener">鄂ICP备20000142号</a> |&nbsp;&nbsp;
      主题 <a href="https://github.com/giscafer/hexo-theme-cafe/" target="_blank">Cafe</a>&nbsp;|&nbsp;&nbsp;
	  <span id="busuanzi_container_site_uv">本站有效访客数<span id="busuanzi_value_site_uv"></span>人</span>
	  <span id="busuanzi_container_site_pv" >| 总访问量 <span id="busuanzi_value_site_pv"></span> 次 </span>
	  <div style="width:300px;margin:0 auto; padding:20px 0;"><a target="_blank" href="http://www.beian.gov.cn/portal/registerSystemInfo?recordcode=42010302002171"style="display:inline-block;text-decoration:none;height:20px;line-height:20px;"><img src="http://www.tzxcode.cn/wp-content/uploads/2020/01/备案图标.png" style="float:left;"/><p style="float:left;height:20px;line-height:20px;margin: 0px 0px 0px 5px; color:#939393;">鄂公网安备 42010302002171号</p></a>
		 	</div>
    </div>
     <div id="footer-right">
      联系方式&nbsp;|&nbsp;1160569243@qq.com
    </div>
	
  </div>
</footer>
 
<script src="/tzxblog/jquery/jquery.min.js"></script>

 <script>
$(document).ready(function() {

    var int = setInterval(fixCount, 50);  // 50ms周期检测函数
    var countOffset = 20000;  // 初始化首次数据

    function fixCount() {            
       if (document.getElementById("busuanzi_container_site_pv").style.display != "none")
        {
            $("#busuanzi_value_site_pv").html(parseInt($("#busuanzi_value_site_pv").html()) + countOffset); 
            clearInterval(int);
        }                  
        if ($("#busuanzi_container_site_pv").css("display") != "none")
        {
            $("#busuanzi_value_site_uv").html(parseInt($("#busuanzi_value_site_uv").html()) + countOffset); // 加上初始数据 
            clearInterval(int); // 停止检测
        }  
    }
       	
});
</script> 
    </div>
    <nav id="mobile-nav">
  
    <a href="/tzxblog/" class="mobile-nav-link">首页</a>
  
    <a href="/tzxblog/shuoshuo/" class="mobile-nav-link">说说</a>
  
    <a href="/tzxblog/archives/" class="mobile-nav-link">归档</a>
  
    <a href="/tzxblog/collections/" class="mobile-nav-link">导航</a>
  
    <a href="/tzxblog/download/" class="mobile-nav-link">资源</a>
  
    <a href="/tzxblog/about/" class="mobile-nav-link">简历</a>
  
</nav>
    <img class="back-to-top-btn" src="/images/fly-to-top.png"/>
<script>
// Elevator script included on the page, already.
window.onload = function() {
  var elevator = new Elevator({
    selector:'.back-to-top-btn',
    element: document.querySelector('.back-to-top-btn'),
    duration: 1000 // milliseconds
  });
}
</script>
      

  

  







<!-- author:forvoid begin -->
<!-- author:forvoid begin -->

<!-- author:forvoid end -->

<!-- author:forvoid end -->



 
<script src="/tzxblog/js/is.js"></script>



  
<link rel="stylesheet" href="/tzxblog/fancybox/jquery.fancybox.css">

  
<script src="/tzxblog/fancybox/jquery.fancybox.pack.js"></script>




<script src="/tzxblog/js/script.js"></script>


<script src="/tzxblog/js/elevator.js"></script>

  </div>
</body>
</html>